TSONN: Time-stepping-oriented neural network for solving partial differential equations

Deep neural networks (DNNs), especially physics-informed neural networks (PINNs), have recently become a new popular method for solving forward and inverse problems governed by partial differential equations (PDEs). However, these methods still face challenges in achieving stable training and obtaining correct results in many problems, since minimizing PDE residuals with PDE-based soft constraint make the problem ill-conditioned. Different from all existing methods that directly minimize PDE residuals, this work integrates time-stepping method with deep learning, and transforms the original ill-conditioned optimization problem into a series of well-conditioned sub-problems over given pseudo time intervals. The convergence of model training is significantly improved by following the trajectory of the pseudo time-stepping process, yielding a robust optimization-based PDE solver. Our results show that the proposed method achieves stable training and correct results in many problems that standard PINNs fail to solve, requiring only a simple modification on the loss function. In addition, we demonstrate several novel properties and advantages of time-stepping methods within the framework of neural network-based optimization approach, in comparison to traditional grid-based numerical method. Specifically, explicit scheme allows significantly larger time step, while implicit scheme can be implemented as straightforwardly as explicit scheme

Quadratic Discriminant Analysis Revisited

In this thesis, we revisit quadratic discriminant analysis (QDA), a standard classification method. Specifically, we investigate the parameter estimation and dimension reduction problems for QDA. Traditionally, the parameters of QDA are estimated generatively; that is the parameters are estimated by maximizing the joint likelihood of observations and their labels. In practice, classical QDA, though computationally efficient, often underperforms discriminative classifiers, such as SVM, Boosting methods, and logistic regression. Motivated by recent research on hybrid generative/discriminative learning, we propose to estimate the parameters of QDA by minimizing a convex combination of negative joint log-likelihood and negative conditional log-likelihood of observations and their labels. For this purpose, we propose an iterative majorize-minimize (MM) algorithm for classifiers of which conditional distributions are from the exponential family; in each iteration of the MM algorithm, a convex optimization problem needs to be solved. To solve the convex problem specially derived for QDA, we propose a block-coordinate descent algorithm that sequentially updates the parameters of QDA; in each update, we present a trust region method for solving optimal estimations, of which we have closed form solutions in each iteration. Numerical experiments show: 1) the hybrid approach to QDA is competitive with, and in some cases significant better than other approaches to QDA, SVM with polynomial kernel ($d=2$) and logistic regression with linear and quadratic features; 2) in many cases, our optimization method converges faster to equal or better optimums than the conjugate gradient method used in the literature. Dimension reduction methods are commonly used to extract more compact features in the hope to build more efficient and possibly more robust classifiers. It is well known that Fisher\u27s discriminant analysis generates optimal lower dimensional features for linear discriminant analysis. However, ...for QDA, where so far there has been no universally accepted dimension-reduction technique in the literature\u27\u27, though considerable efforts have been made. To construct a dimension reduction method for QDA, we generalize the Fukunaga-Koontz transformation, and propose novel affine feature extraction (AFE) methods for binary QDA. The proposed AFE methods have closed-form solutions and thus can be solved efficiently. We show that 1) the AFE methods have desired geometrical, statistical and information-theoretical properties; and 2) the AFE methods generalize dimension reduction methods for LDA and QDA with equal means. Numerical experiments show that the new proposed AFE method is competitive with, and in some cases significantly better than some commonly used linear dimension reduction techniques for QDA in the literature

Phylogenetic and evolutionary analysis of the septin protein family in metazoan

AbstractSeptins, a conserved family of cytoskeletal GTP-binding proteins, were presented in diverse eukaryotes. Here, a comprehensive phylogenetic and evolutionary analysis for septin proteins in metazoan was carried out. First, we demonstrated that all septin proteins in metazoan could be clustered into four subgroups, and the representative homologue of every subgroup was presented in the non-vertebrate chordate Ciona intestinalis, indicating that the emergence of the four septin subgroups should have occurred prior to divergence of vertebrates and invertebrates, and the expansion of the septin gene number in vertebrates was mainly by the duplication of pre-existing genes rather than by the appearance of new septin subgroup. Second, the direct orthologues of most human septins existed in zebrafish, which suggested that human septin gene repertoire was mainly formed by as far as before the split between fishes and land vertebrates. Third, we found that the evolutionary rate within septin family in mammalian lineage varies significantly, human SEPT1, SEPT 10, SEPT 12, and SEPT 14 displayed a relative elevated evolutionary rate compared with other septin members. Our data will provide new insights for the further function study of this protein family

A Multi-Criteria Decision-Making Scheme for Multi-Aircraft Conflict Resolution

Multi-Aircraft Conflict Resolution (MACR) is a Multi-Criteria Decision-Making (MCDM) problem, which involves multiple stakeholders (airline, air traffic controller, and aircraft) with competing and incommensurable objectives. This paper proposes a two-step MCDM scheme to the solution of MACR. In the first step, a second order cone program is adopted to generate a set of candidate resolution strategies with different minimum separations between trajectories. Each candidate strategy is then evaluated via three criteria modeling the interests of the stakeholders. In the second step, the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) approach is used to determine the best strategy that realizes an adequate tradeoff among the competing interests while coping with their incommensurability. Some numerical results are presented to show the efficacy of the proposed scheme. Interestingly, the minimum separations associated with the best resolution strategies according to either the interest of the airline or that of the aircraft both differ from the one adopted in the current air traffic control operation

Distributed Cooperative Regulation for Multiagent Systems and Its Applications to Power Systems: A Survey

Cooperative regulation of multiagent systems has become an active research area in the past decade. This paper reviews some recent progress in distributed coordination control for leader-following multiagent systems and its applications in power system and mainly focuses on the cooperative tracking control in terms of consensus tracking control and containment tracking control. Next, methods on how to rank the network nodes are summarized for undirected/directed network, based on which one can determine which follower should be connected to leaders such that partial followers can perceive leaders’ information. Furthermore, we present a survey of the most relevant scientific studies investigating the regulation and optimization problems in power systems based on distributed strategies. Finally, some potential applications in the frequency tracking regulation of smart grids are discussed at the end of the paper